PIM: Video Coding using Perceptual Importance Maps

Human perception is at the core of lossy video compression, with numerous approaches developed for perceptual quality assessment and improvement over the past two decades. In the determination of perceptual quality, different spatio-temporal regions of the video differ in their relative importance to the human viewer. However, since it is challenging to infer or even collect such fine-grained information, it is often not used during compression beyond low-level heuristics. We present a framework which facilitates research into fine-grained subjective importance in compressed videos, which we then utilize to improve the rate-distortion performance of an existing video codec (x264). The contributions of this work are threefold: (1) we introduce a web-tool which allows scalable collection of fine-grained perceptual importance, by having users interactively paint spatio-temporal maps over encoded videos; (2) we use this tool to collect a dataset with 178 videos with a total of 14443 frames of human annotated spatio-temporal importance maps over the videos; and (3) we use our curated dataset to train a lightweight machine learning model which can predict these spatio-temporal importance regions. We demonstrate via a subjective study that encoding the videos in our dataset while taking into account the importance maps leads to higher perceptual quality at the same bitrate, with the videos encoded with importance maps preferred $1.8 \times$ over the baseline videos. Similarly, we show that for the 18 videos in test set, the importance maps predicted by our model lead to higher perceptual quality videos, $2 \times$ preferred over the baseline at the same bitrate

Data Compression versus Signal Fidelity Trade-off in Wired-OR ADC Arrays for Neural Recording

This paper investigates the efficacy of a wired-OR compressive readout architecture for neural recording, which enables simultaneous data compression of action potential signals for high channel count electrode arrays. We consider a range of wiring configurations to assess the trade-offs between compression ratio and various task-specific signal fidelity metrics. We consider the fidelity in threshold crossing detection, spike assignment, and waveform estimation, and find that for an event SNR of 7-10 the readout captures at least 80% of the spike waveforms at ∼150x data compression.Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Bio-Electronic

Illusion of large on-chip memory by networked computing chips for neural network inference

International audienc

14.3 A 43pJ/cycle non-volatile microcontroller with 4.7μs shutdown/wake-up integrating 2.3-bit/cell resistive RAM and resillence techniques

Non-volatility is emerging as an essential on-chip memory characteristic across a wide range of application domains, from edge nodes for the Internet of Things (IoT) to large computing clusters. On-chip non-volatile memory (NVM) is critical for low-energy operation, real-time responses, privacy and security, operation in unpredictable environments, and fault-tolerance [1]. Existing on-chip NVMs (e.g., Flash, FRAM, EEPROM) suffer from high read/write energy/latency, density, and integration challenges [1]. For example, an ideal IoT edge system would employ fine-grained temporal power gating (i.e., shutdown) between active modes. However, existing on-chip Flash can have long latencies (> 23 ms latency for erase followed by write), while inter-sample arrival times can be short (e.g., 2ms in [2]).Accepted versionWork supported in part by DARPA, NSF/NRI/GRC E2CDA, and the Stanford SystemX Alliance